Quality metrics for detailed clinical models

OBJECTIVE To develop quality metrics for detailed clinical models (DCMs) and test their validity. METHODS Based on existing quality criteria which did not include formal metrics, we developed quality metrics by applying the ISO/IEC 9126 software quality evaluation model. The face and content validity of the initial quality metrics were assessed by 9 international experts. Content validity was defined as agreement by over 70% of the panelists. For eliciting opinions and achieving consensus of the panelists, a two round Delphi survey was conducted. Valid quality metrics were considered reliable if agreement between two evaluators' assessments of two example DCMs was over 0.60 in terms of the kappa coefficient. After reliability and validity were tested, the final DCM quality metrics were selected. RESULTS According to the results of the reliability test, the degree of agreement was high (a kappa coefficient of 0.73). Based on the results of the reliability test, 8 quality evaluation domains and 29 quality metrics were finalized as DCM quality metrics. CONCLUSION Quality metrics were validated by a panel of international DCM experts. Therefore, we expect that the metrics, which constitute essential qualitative and quantitative quality requirements for DCMs, can be used to support rational decision-making by DCM developers and clinical users.